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PREFACE 


This  technical  memorandum  describes  the  multimedia  authoring  procedures  and  lessons 
learned  during  the  publication  of  AFFTC-TR-97-49,  Evaluation  of  the  AN/AVS-8  (V)  Night 
Vision  System,  on  compact  disc  read-only  memory  (CD-ROM).  This  memorandum  identifies 
potential  benefits  of  electronic  publication,  the  potential  of  multimedia  to  be  an  integral  part  of 
the  data  collection  and  analysis  process,  the  specific  procedures  employed  for  CD-ROM 
development,  and  includes  recommendations  for  future  Air  Force  Flight  Test  Center  publishing. 
In  addition  to  this  memorandum,  the  above-identified  technical  report  on  CD-ROM  served  as  an 
example  of  multimedia  publication. 

Sincere  appreciation  is  expressed  to  Annette  Anderson  (412  TW/TSFR)  and  Kurt  Joseph 
(412TW/RMSP)  for  their  many  valuable  technical  contributions  to  this  effort. 
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EXECUTIVE  SUMMARY 

In  1997,  the  Air  Force  Flight  Test  Center  (AFFTC)  conducted  an  evaluation  of  the  new  Night 
Vision  System  [AN/AVS-8  (V)]  for  the  Human  Systems  Center  (now  the  311  Human  Systems 
Test  Wing  at  Brooks  AFB,  Texas).  The  results  of  this  evaluation  were  sent  to  the  customer  in  the 
form  of  a  paper-based  test  and  evaluation  results  sheet  (TERS),  AFFTC-TERS-97-17,  in  June 
1998.  After  delivery  of  the  TERS,  the  customer  indicated  that  they  would  also  like  this  report  on 
compact  disc-read-only  memory  (CD-ROM).  The  AFFTC  developed  a  CD-ROM  version  of  the 
TERS  using  Microsoft  Word®  for  the  text,  hyperlinks  to  facilitate  navigation  within  the 
document,  and  high-resolution  picture  files  of  the  illustrations.  Prior  to  this  report  publication  on 
CD-ROM,  all  AFFTC  reports  were  paper-based  publications.  The  CD-ROM  version  was  well 
received  by  the  customer. 

The  412  Test  Wing,  Airframe  Systems  Integration  Division  (TSS)  decided  to  fund  the 
development  of  the  TERS  into  a  multimedia  technical  report  (TR).  The  objective  of  this  project 
was  to  demonstrate  multimedia  methods  for  data  collection  and  analysis  and  to  develop  an 
example  multimedia  technical  report  on  CD-ROM  with  an  accompanying  hardcopy  version.  The 
electronic  product  needed  to  be  in  a  format  acceptable  to  the  Defense  Technical  Information 
Center  (DHC)  for  electronic  publishing.  The  electronic  report  was  developed  in  Adobe  Portable 
Document  Format  (PDF)®,  extensive  hyperlinks  were  used  to  facilitate  navigation  throughout  the 
document.  An  interactive  multimedia  presentation  was  developed  in  IMSI  Multimedia  Fusion® 
as  a  self-running  executable  file.  High-resolution  picture  files  of  the  illustrations  and  multimedia 
frames  were  included  for  printing  or  detailed  visual  inspection. 

The  overall  objective  of  demonstrating  the  application  and  worth  of  multimedia  methods  and 
electronic  publication  for  testing  and  reporting  was  met.  The  developed  TR  was  published  on 
CD-ROM  in  a  format  that  was  acceptable  to  DTIC  for  electronic  publishing.  The  multimedia 
technical  report  also  was  well  received  by  the  customer.  Electronic  publishing  on  CD-ROM, 
including  multimedia,  should  be  considered  for  future  AFFTC  technical  publications. 

Many  lessons  were  learned  during  this  effort,  and  these  lessons  are  included  as  an  appendix 
to  this  technical  memorandum.  Multimedia  should  be  considered  during  the  initial  test-planning 
phase  to  ensure  success  with  data  collection,  data  analysis,  and  results  publication.  No  single 
approach  considered  for  text  and  multimedia  authoring  met  all  requirements,  and  compromises 
were  inherent  in  each  approach  selected.  The  PDF  format  should  be  considered  for  electronic 
publishing  of  text  files  because  the  Acrobat  viewer  facilitated  navigation  through  the  document 
and  retained  the  original  format  for  viewing  as  well  as  printing.  The  use  of  self-running 
multimedia  files  should  be  used  because  self-running  files  did  not  require  installation  on  the 
user’s  computer,  and  multimedia  display  was  not  dependent  upon  the  specific  browser  or  browser 
settings  on  the  user’s  computer. 
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INTRODUCTION 


BACKGROUND 

In  1997,  the  Air  Force  Flight  Test  Center  (AFFTC)  conducted  an  evaluation  of  a  night  vision 
system  for  the  Human  Systems  Center.  This  new  night  vision  system  (NVS)  could  automatically 
separate  the  night  vision  goggles  (NVGs)  from  the  user’s  helmet  at  the  start  of  an  ejection 
sequence.  The  results  of  this  report  were  sent  to  the  customer  in  the  form  of  a  paper-based  test 
and  evaluation  results  sheet  (TERS),  AFFTC-TERS-97-17,  in  January  1998  (Reference  1).  After 
delivery  of  the  TERS,  the  customer  requested  a  copy  of  the  report  on  compact  disc-read-only 
memory  (CD-ROM).  The  AFFTC  developed  a  CD-ROM  version  of  the  TERS  using  Microsoft 
Word®  for  the  text,  and  which  included  hyperlinks  to  facilitate  navigation  within  the  document 
and  high-resolution  picture  files  of  the  illustrations.  The  CD-ROM  version  of  the  TERS  was 
delivered  in  June  1998,  and  was  well  received  by  the  customer. 

Prior  to  the  delivery  of  the  TERS  on  CD-ROM,  technical  reports  at  the  AFFTC  had  been 
published  as  bound  hardcopy.  The  AFFTC  author’s  guide  addresses  technical  reports  primarily  as 
paper-based  documents  (Reference  2).  Paper-based  publishing  had  a  history  of  reader  acceptance 
but  had  disadvantages  compared  with  electronic  publishing,  that  is,  publishing  on  computer 
media.  To  demonstrate  the  potential  of  electronic  publication,  the  412  Test  Wing,  Airframe 
Systems  Integration  Division  (TSS)  decided  to  fund  the  development  of  the  content  of  the  TERS 
as  a  multimedia  technical  report  (TR)  to  be  published  on  CD-ROM.  Although  the  entire  report 
could  have  been  developed  as  a  single  multimedia  presentation,  the  project  was  developed  as  two 
distinct  parts,  a  text  report  and  supporting  multimedia  presentation,  published  on  CD-ROM. 

PROJECT  OBJECTIVES 

The  overall  objective  of  this  project  is  to  demonstrate  the  application  and  value  of  multimedia 
methods  and  electronic  publication  for  testing  and  reporting.  Specific  project  objectives  are: 

1.  To  employ  multimedia  as  part  of  the  analytic  process  to  support  assessment  of  test 
objectives,  conclusions,  and  recommendations, 

2.  To  exercise  multimedia  procedures  that  would  support: 

a.  Frame-by-frame  (shuttered)  video  presentations, 

b.  Determination  of  time-space  positioning  information  (TSPI),  and 

c.  Synchronized  display  of  data  plots  and  movie  frames, 

3 .  To  provide  zoom-in  display  capability  for  the  user, 

4.  To  determine  a  report  format  that  would  be  compatible  with  a  majority  of  user  computers 
with  a  minimum  of  requirements, 

5.  To  determine  a  report  format  that  would  meet  requirements  of  the  Defense  Technical 
Information  Center  (DTIC),  and 

6.  To  produce  a  final  product  that  is  durable  and  sustainable  over  time. 
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ELECTRONIC  PUBLISHING 


A  comparison  of  paper  and  electronic  publication  methods  is  presented  in  Table  1.  Writing 
styles  and  techniques  may  be  adopted  to  maximize  the  new  capabilities  of  electronic  publication 
(References  3,  4,  5  and  6).  An  extremely  powerful  new  authoring  capability  for  electronic 
publication  was  the  hyperlink.  Authors  could  ‘hot  spot’  individual  words  or  pictures  so  that 
mouse  clicks  on  these  ‘hot  spots’  would  transfer  (hyperlink)  the  reader  instantly  to  different 
sections  within  the  document  or  anywhere  in  the  world  via  the  World  Wide  Web  (WWW). 
Another  electronic  publication  capability  was  the  ‘mouse-over,’  or  movement  of  the  cursor  over 
an  object  or  word  usually  achieved  by  mouse  movement.  A  mouse-over  could  make  words  or 
pictures  change  in  some  way  when  the  cursor  was  moved  over  them.  For  example,  a  mouse-over 
could  turn  a  word  red,  and  a  mouse  click  on  this  red  word  could  cause  some  additional  action, 
such  as  a  narration,  a  movie  presentation,  or  hyperlink  to  additional  text.  Another  electronic 
publication  capability  was  to  make  the  table  of  contents  available  as  ‘thumbnails’  (tiny  pictures  of 
each  document  page).  Additionally,  it  was  possible  to  add  features  into  multimedia  documents, 
such  as  animations,  moving  or  flashing  words,  and  fancy  transitions  from  one  frame  to  the  next. 
However,  these  techniques  should  be  used  to  support  communication  of  the  message  and  should 
not  be  used  where  they  might  detract  from  the  message  itself. 


Table  1 

DIFFERENCES  BETWEEN  PUBLISHING  APPROACHES 


Paper-basec 

Documents 

Pros 

Cons 

Long  tradition  of  successful  use  and  acceptance. 

Expensive  to  produce,  ship,  and  bulky  to  store. 

No  support  equipment  required  for  viewing. 

To  copy  document,  disassembly  could 
be  required  and  image  quality  degraded 
with  each  generation. 

Can  be  published  with  archival 
quality  materials. 

Color  was  an  extra  publishing  and 
copying  expense. 

Portable. 

Cannot  convert  to  electronic  form  easily  as 
optical  character  recognition  was  difficult 
and  error  prone. 

Electronic 

Documents 

Pros 

Cons 

Permits  user  control  and  interactivity.  User 
can  select  instantly  what  to  read,  see,  or  hear. 
User  can  select  presentation  format. 

User  must  have  computer  supporting 
multimedia  to  experience  full  capability 
of  document. 

Hyperlinks  could  jump  within  and  between 
documents,  including  documents 
on  the  World  Wide  Web  (making 
documents  unbounded). 

Current  typical  desktop  computers  may  not 
be  adequate  for  electronic  documents. 

Full  color,  movies,  sound  and  animations 
can  be  included  without  additional 
publishing  cost. 

Potential  obsolescence  of  required  computer 
playback  software  and  hardware. 

An  entire  report  could  be  put  on  one 
compact  disc. 

Some  paper-based  copies  may  still 
be  needed. 
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MULTIMEDIA 


Multimedia  was  defined  as  a  single  presentation  that  included  several  mediums  of 
communication,  such  as  text  documents,  movies,  light  shows,  still  slides,  and  narration.  Media 
that  were  considered  for  this  project  had  to  be  publishable  on  CD-ROM.  Multimedia  offered 
additional  potential  as  an  instrument  for  data  collection,  data  analysis,  and  the  presentation  of 
results.  By  employing  computer-assisted  modeling  and  simulation,  multimedia  methods  could 
provide  test  engineers  with  clearer  pictures  of  test  events.  The  same  media  products  collected 
during  testing  and  simulation  studies  could  be  carried  over  to  the  final  report,  thus  adding 
minimal  burden  or  time  for  report  publication.  Additionally,  multimedia  could  offer  improved 
communication  for  technical  issues  and  test  results.  Multimedia  presentations  could  be  authored 
to  permit  interaction  between  the  user  and  the  presentation  media,  such  as  the  topics  to  read,  the 
movies  and  animated  sequences  to  see,  the  narration  to  hear,  and  the  sequence  of  report 
presentation.  User  involvement  through  multimedia  interaction  has  been  known  to  increase 
motivation  and  learning  (Reference  7).  Furthermore,  multimedia  documents  could  be  structured 
from  a  ‘top-down’  perspective,  so  that  the  reader  could  focus  on  top-level  issues  and  then  go  into 
greater  detail  (analysis).  Conversely,  the  reader  could  start  with  specific  details  and  work  from  the 
details  to  the  major  findings  (synthesis). 
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PROJECT  ELEMENTS 


INITIAL  PROJECT  PLAN  AND  PROBLEMS  ENCOUNTERED 

Initial  Plan: 

The  initial  plan  for  the  expected  multimedia  project  is  shown  in  Figure  1.  Multimedia  was  to 
be  employed  as  part  of  the  analytic  process  to  assess  specific  test  objectives.  To  meet  project 
objectives,  this  plan  relied  heavily  on  movie  film  digitization  and  data  analyses  from  the  Range 
(412  TW/TSR).  The  Range  had  a  capability  called  the  Edwards  Store  Separation  Analysis  System 
(ESS AS)  which  could  support  modeling  and  simulation  studies  of  events  captured  on  16-mm 
motion  picture  film.  The  initial  project  plan  called  for  the  ESSAS  group  to  use  the  16-mm  sled 
test  movie  film  in  conjunction  with  wire  frame  models  of  night  vision  system  (NVS)  hardware  to 
compute  TSPI  measurements,  such  as  NVG  goggle  rotation,  goggle-head  miss-distance,  and 
goggle  trajectory.  Data  plots  would  be  time-synchronized  to  associated  movie  files  on  a  frame  by 
frame  basis.  Capability  was  to  be  provided  to  permit  users  to  zoom  into  areas  of  special  interest. 
Sounds  and  narration  were  to  be  used  to  clarify  events  of  interest. 


Notes:  1 .  NVS  -  night  vision  system 

2.  CD-ROM  -  compact  disk-read-only  memory 


Figure  1  Initial  Project  Plan 
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Problems  Encountered: 


The  High-speed  Test  Track  at  Holloman  AFB,  New  Mexico  was  requested  to  provide  still 
photographs  and  high-speed  motion  pictures  of  the  NVS  test  for  visual  analysis.  Topics  of 
concern  for  the  NVS  test  were  the  operation  of  the  automatic  goggle  release  system,  the 
trajectory  of  the  goggles  upon  release,  differences  between  aircraft  types,  and  differences 
between  single  seat  and  tandem  seat  aircraft.  The  Test  Track  was  not  requested  to  provide 
data  suitable  for  ESSAS  analyses,  which  would  have  included  triangulated  camera  positions, 
precise  camera  positioning,  and  calibration  of  the  photo-optical  systems.  Consequently,  the 
data  collected  during  the  NVS  test  were  inadequate  to  support  ESSAS  analyses  or  to 
demonstrate  multimedia  as  part  of  the  analytic  processes.  If  Edwards  Store  Separation 
Analysis  System  (ESSAS)  analysis  is  planned,  ESSAS  requirements  should  be 
considered  during  the  test-planning  phase.  (LL1)1  The  ESSAS  data  requirements  are 
presented  in  Reference  8  and  Appendix  A.  The  lessons  learned  throughout  this  project  are 
documented  in  Appendix  B.  Because  the  data  collected  during  the  NVS  test  were  inadequate 
to  support  ESSAS  analyses,  this  project  was  unable  to  demonstrate  multimedia  as  part  of  the 
analytic  processes.  The  data  collected  during  the  NVS  test  were  inadequate  to  support  ESSAS 
analyses  because  multimedia  data  analysis  had  not  been  identified  in  the  initial  project  plan. 
Multimedia  should  be  considered  during  the  initial  test-planning  phase  to  ensure 
success  with  data  collection,  data  analysis,  and  results  publication.  (Rl)2 

Initially,  the  ESSAS  group  was  to  digitize  selected  NVS  test  motion  picture  films  for 
conversion  into  high-resolution  (640  X  480  pixels)  digital  movie  files.  However,  the  ESSAS 
group's  camera  was  not  color  capable  and  the  digital  output  file  was  non-standard  in  format. 
Therefore,  the  ESSAS  group  could  not  perform  the  film  digitization  as  identified  in  the  initial 
project  plan  (Reference  9).  Additionally,  the  project  could  not  achieve  the  following 
objectives:  obtaining  shuttered  video  for  the  multimedia,  determination  of  TSPI  information, 
and  the  synchronization  of  data  plots  and  movie  frames.  The  Range  was  in  the  process  of 
upgrading  their  system  with  a  color  digital  camera  having  a  standard  file  format.  The 
upgraded  ESSAS  capability  should  be  considered  as  a  resource  to  support  multimedia 
data  collection,  film  digitization,  and  data  analysis.  (LL2) 

ACTUAL  PROJECT  EXECUTION 

Figure  2  shows  the  actual  project  plan,  which  was  used  for  editing,  authoring,  and 
publishing  the  technical  report  on  CD-ROM.  The  hardware  and  software  used  for  this  project 
are  identified  in  Appendix  C.  Since  multimedia  techniques  were  not  used  to  assess 
test  objectives,  a  decision  was  made  to  build  the  multimedia  presentation  upon  the  NVS  sled 
test  runs. 


1  Numerals  preceded  by  an  LL  within  parentheses  at  the  end  of  a  sentence  correspond  to  the  lessons  learned  tabulated  in  Appendix  B 
of  this  technical  memorandum. 

2Numerals  preceded  by  an  R  within  parentheses  at  the  end  of  a  sentence  correspond  to  the  recommendations  tabulated  in  the 
Conclusions  and  Recommendations  section  of  this  memorandum. 
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Identify  Multimedia 


Figure  2  Actual  Project  Execution 


Authoring  Tools  Considered; 


Table  2  identifies  alternatives  for  text  authoring  and  formatting  for  publication.  No  single 
approach  met  all  text  and  multimedia  authoring  requirements,  and  the  approaches  selected 
involved  compromises.  Requirements  for  DTIC  electronic  publications  are  identified  in 
Reference  10.  Approaches  were  evaluated  on  how  well  they  met  the  following  criteria: 

a.  Acceptable  to  DTIC, 

b.  Usable  on  most  multimedia  computers, 

c.  Moderate  hardware  requirements  for  user  computer, 

d.  Minimum  requirements  to  install  software  on  user  computer, 

e.  Intuitive  and  easy  to  use,  and 

f.  Capable  of  printing  document  without  altering  format  or  pagination. 


Table  2 

TEXT  AUTHORING  TOOLS  CONSIDERED 


Authoring  Tool 

Pros 

Cons 

Microsoft 

Word® 

Format  retained  when  copied  and 
pasted  into  Word  documents. 

User  computer  was  required  to  have 
Microsoft  Word. 

Navigation  throughout  document  was 
difficult. 

Display  or  printed  output  may  be 
altered  when  using  different  printer 
drivers. 

Standard 

Generalized 

Markup 

Language 

(SGML) 

At  one  time,  SGML  was  a  DoD 
standard  for  cross-platform 
publication. 

In  practice,  SGML  was  never 
implemented  by  DoD. 

Infrequently  used  and  costly. 

Not  recommended  by  Defense 
Technical  Information  Center. 

Static  Hypertext 
Mark-up 

Language 

(HTML) 

Most  web  browsers  supported  static 
HTML. 

Navigation  navigate  through 
document  was  difficult. 

Display  was  browser  dependent. 

Adobe  Portable 
Document 

Format  (PDF)® 

Free  viewer,  already  installed  on 
most  computers. 

Navigation  within  and  between  files 
was  supported. 

Very  convenient  and  easy  to  use. 
Format  and  pagination  remained 
invariant  for  display  and  printing. 

Formatting  lost  when  copying  and 
pasting  into  Microsoft  Word. 

7 


Since  AFFTC  TRs  were  distributed  to  DUC,  the  electronic  publishing  approaches  selected  must 
meet  DUC  requirements.  Several  computer  document  formats  were  considered  for  electronic 
publication.  The  standard  generalized  mark-up  language  (SGML)  was  specified  as  a  DoD  standard  for 
cross-platform  publication,  but  SGML  was  never  implemented,  the  software  authoring  programs  were 
expensive,  and  the  overall  approach  was  not  recommended  by  DTIC.  Another  possibility  was  static 
HTML,  but  it  suffered  in  terms  of  being  browser-dependent,  providing  poor  navigation  throughout  the 
document,  and  loosing  formatting  when  printed.  Microsoft  Word  was  considered,  but  Word  was  not 
well  suited  for  computer  display  since  the  printer  driver  selected  on  the  user’s  computer  could  alter  the 
visual  display  format,  printing  format,  and  pagination.  The  Adobe  PDF  Version  3.01  was  considered. 
Microsoft  Word,  HTML,  and  Adobe  PDF  formats  were  acceptable  to  DUC  for  electronic  publishing. 
No  single  approach  considered  for  text  and  multimedia  authoring  met  all  requirements,  and 
compromises  were  inherent  in  each  approach  selected.  (LL3)  The  Adobe  PDF  format  was 
selected  as  the  most  desirable  format  for  electronic  publication  because  it  retained  both  display  and 
printing  format  and  because  the  viewer  included  additional  features  to  facilitate  document  browsing 
on  the  user’s  computer.  This  approach  did  require  that  an  Adobe  Acrobat  Viewer  was  installed  on  the 
user’ s  computer,  but  the  viewer  was  freely  distributed  and  in  widespread  use. 

High-resolution  picture  files  were  usually  used  in  documents  intended  for  printing  because  they 
provided  superior  printing  quality.  However,  high-resolution  picture  files  provided  annoyingly  slow 
computer  display  while  low-resolution  picture  files  could  provide  faster  and  more  satisfactory 
computer  display.  High-resolution  picture  files  could  be  used  for  detailed  visual  examination, 
projection  during  group  meetings,  photo  editing,  quality  printouts,  or  for  incorporation  into  other 
documents.  Low-resolution  picture  files  should  be  used  within  documents  to  provide  rapid 
display  and  high-resolution  copies  of  these  pictures  provided  on  the  compact  disc-read-only 
memory  for  printing,  high-quality  display,  or  other  uses.  (LL4) 

Publishing  for  electronic  media  was  not  the  same  as  publishing  on  paper.  Conventional  paper- 
based  documents  used  blank  pages  so  that  a  major  topic  always  started  on  die  right  side  of  a  center- 
folded  document.  This  approach  was  logical  and  helpful  for  the  reader.  Blank  pages  should  not  be 
incorporated  into  electronic  publications  because  they  are  annoying  for  the  reader.  (I  T  S) 
Presented  with  a  blank  page,  the  computer  user  had  to  scroll  down  or  up  with  the  mouse  or  keyboard 
to  find  meaningful  document  material.  Therefore  blank  pages  were  eliminated  from  the  documents 
intended  for  computer  display.  Dual  column  format  was  used  for  printed  documents  to  facilitate 
reading.  Dual  column  format  required  less  horizontal  eye  movement  per  line.  Single  column  format 
should  be  used  for  electronic  publications  because  dual  column  format  required  twice  as  much 
mouse  or  keyboard  scrolling,  which  was  annoying  for  the  reader.  (LL6)  Therefore,  single 
column  document  format  was  selected  for  documents  intended  for  the  electronic  publication. 

Only  minor  text  touch-up  was  available  within  Acrobat  Exchange  for  editing  the  PDF  files. 
Editing  in  Acrobat  Exchange  was  restricted  to  editing  words  on  a  single  line,  as  word  wrapping  was 
not  possible.  All  text  editing  (except  for  inserting  or  revising  hyperlinks)  should  be  done  in  the 
Microsoft  Word  files  before  conversion  into  PDF  files.  (LL7) 

Minimal  effort  was  required  to  convert  Microsoft  Word  files  to  Adobe  PDF.  The  conversion  of 
Word  files  to  PDF  files  was  accomplished  using  a  suite  of  Adobe  Acrobat  Version  3.01  programs, 
including  the  Acrobat  PDF  Writer,  Acrobat  Distiller,  and  Acrobat  Exchange  in  conjunction  with  a 
PostScript®  printer  driver  (a  QMS  Magicolor  Plus  Level  2  driver  was  used  for  this  project).  Acrobat 
Version  3.01  was  difficult  to  setup,  and  therefore  the  important  settings  used  for  PDF  file 
development  during  this  project  were  provided  in  Appendix  D.  Adobe  Acrobat  Version  4.0  was  found 
to  be  much  easier  to  install  and  set-up  than  Version  3.01.  Adobe  Acrobat  Version  4.0  or  later 
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should  be  used  for  creating  PDF  files  from  document  files.  (LL8)  The  Adobe  programs  were 
initiated  in  an  automated  sequence  from  within  Microsoft  Word  by  using  the  ‘Create  Adobe  PDF’ 
command  from  the  file  menu.  During  the  early  stages  of  this  project,  the  hyperlinks  within  the 
Word  document  were  changed  to  reference  the  anticipated  Adobe  PDF  files.  However,  the  Adobe 
PDF  Writer  Version  3.01  did  not  carry  over  linking  to  bookmarks  thus  requiring  additional 
editing  using  Acrobat  Exchange.  Hyperlinks  should  be  established  in  the  Adobe  PDF  files  as 
they  are  configured  for  publication  to  reduce  redundancy  of  verification  efforts.  (LL9) 
Initially,  hyperlinks  were  set  to  the  start  of  a  text  file,  but  this  was  inconvenient  for  the  reader. 
However,  hyperlinks  could  be  set  to  a  file  and  then  to  bookmarks  within  the  file  for  precise 
location.  Hyperlinks  should  take  the  reader  to  the  exact  position  desired  within  a  document, 
rather  than  to  the  start  of  a  file.  (LL10)  Documents  for  electronic  publication  should  be 
organized  in  the  folder  and  file  arrangements  to  be  recorded  on  the  CD-ROM.  Trial  compact 
discs  should  be  recorded  and  tested  on  different  computers  to  ensure  that  hyperlinks  were 
not  made  to  unintended  files  residing  in  other  locations  on  the  authoring  computer.  (LL11) 

Because  electronic  publications  including  multimedia  were  new  to  the  AFFTC,  many  desktop 
computer  systems  were  not  set-up  adequately  to  support  multimedia.  As  a  prerequisite  for 
successful  electronic  publication,  all  user  computers  should  be  configured  to  support 
multimedia.  (LL12)  Several  alternative  formats  were  considered  for  authoring  multimedia,  and 
these  formats  are  shown  in  Table  3.  The  SGML  format  was  not  selected  because  it  was  not 
implemented  by  DoD  and  was  not  recommended  by  DTIC.  The  HTML  format  was  also 
considered,  but  the  display  of  HTML  documents  depended  heavily  upon  the  web  browser  that 
was  installed  on  the  user’s  computer.  Individual  computer  setups  were  found  to  vary 
considerably  in  terms  of  browser  selection,  browser  plug-ins,  video  cards,  and  display 
settings,  all  of  which  affected  performance.  (LL13) 


Table  3 

MULTIMEDIA  AUTHORING  TOOLS  CONSIDERED 


|  Authoring  Tool 

Pros 

Cons 

Standard 
Generalized 
Markup  Language 
(SGML) 

At  one  time,  SGML  was  a  DoD 
standard  for  cross-platform 
publication. 

Not  implemented  by  DoD. 
Infrequently  used  and  costly. 

Not  recommended  by  DTIC1. 

Static  Hypertest 
Markup  Language 
(HTML) 

Works  with  almost  all  web  browsers. 

Limited  capability. 

Dynamic  HTML 

Many  display  options  were  available. 

Results  varied  on  different  computers. 
Browser  dependent. 

Astound  HTML  failed  validation  test. 

Astound® 
executable  file 

Easy,  intuitive  to  author. 

No  software  had  to  be  installed  on 
user  computer. 

Some  sound  files  within  the 
self-run  product  were  dropped 
through  software  glitch. 

IMSI  Multimedia 
Fusion® 
executable  file 

Highly  capable  and  versatile  program. 
Fast,  stable  output  product. 

User  was  not  required  to  install 
anything  on  computer. 

Selecting  from  a  wide  variety  of 
options  made  authoring  time 
consuming. 

Could  not  copy  and  paste  information 
from  self-running  program. 

‘Defense  Technical  Information  Center 
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An  authoring  program  called  Astound®  was  suggested  by  an  outside  multimedia 
expert.  The  Astound  program  was  easy  to  program  and  could  produce  document  outputs 
as  static  HTML,  dynamic  HTML,  or  as  self-running  files.  However,  when  producing 
executable  self-running  programs.  Astound  occasionally  dropped  some  sound  files. 
Concern  with  the  Astound  program  dropping  sound  files  led  to  the  investigation  of 
a  program  called  IMSI  Multimedia  Fusion®.  No  single  authoring  approach  considered 
for  text  and  multimedia  authoring  met  all  requirements,  and  compromises  were  inherent 
in  each  approach  selected.  IMSI  Multimedia  Fusion  was  selected  as  the  best  of  the 
alternatives  tested  for  multimedia  publishing  because  it  was  capable  as  an  authoring 
tool  and  produced  a  solid,  useful  product  that  ran  on  almost  all  the  computers  tested. 
The  only  problems  that  occurred  when  using  the  Multimedia  Fusion  product  were 
with  the  lack  of  suitable  Motion  Picture  Experts  Group  (MPEG)  movie  viewers  on  the 
user’s  computer. 

Multimedia  Authoring: 

Reference  7  provides  an  excellent  discussion  of  multimedia  production  processes. 
Multimedia  was  developed  on  a  frame-by-frame  basis.  Frame  sizes  for  computer 
display  were  specified  in  pixels.  Common  computer  display  formats  included: 
640  X  480  pixels,  800  X  600  pixels,  and  1024  X  768  pixels.  Note  that  these  formats 
all  had  a  4:3  aspect  ratio,  and  that  consistency  of  aspect  ratio  was  desirable  for  esthetic 
reasons.  An  image  size  of  800  X  600  pixels  was  selected  for  the  multimedia  product 
because  all  multimedia  computers  could  support  this  format  and  the  picture  quality 
was  satisfactory.  This  frame  size  required  a  seventeen-inch  (or  larger)  color  monitor  for 
best  display. 

The  multimedia  presentation  was  conceptualized  in  storyboard  fashion,  similarly  to 
the  approach  used  for  developing  animations  and  movies.  The  multimedia  design 
focused  on  providing  report  content;  in  this  case  the  results  of  the  NVS  sled  test. 
The  presentation  was  developed  as  a  series  of  discrete  frames  and  these  frames 
organized  in  terms  of  logical  flow.  Figure  3  shows  the  structure  of  the  multimedia 
product.  The  presentation  started  with  a  title  or  home  page,  shown  in  this  figure  as 
frame  one.  The  home  page  presented  the  reader  with  the  overall  scope  and  topics  that 
were  included.  The  home  page  had  links  to  take  the  reader  to  any  of  the  major 
document  sections.  The  major  sections  were  designed  to  be  relatively  complete  within 
themselves,  like  chapters  in  a  book.  Within  each  section,  links  were  provided  so  that 
the  reader  could  always  return  to  the  home  page.  Links  to  the  home  page  prevented  the 
reader  from  ending  up  in  dead-end  branches  with  nowhere  to  turn.  Each  major  section 
consisted  of  one  or  more  frames. 
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72 

C  &  R  for 
Sled  Tests 


81 

Side  View 
Movie 


Notes: 


32  -  44  &  79 
Canopy  Opening 
Sequence 


1. NVGs-  night  vision  goggles 

2.  BIT  -  built-in  test 

3.  Numbers  in  each  block  are  frame  (page)  numbers,  where  some  blocks  contain 
more  than  one  frame 

4.  NVS  -  night  vision  system 


Figure  3  Flowchart  Used  for  Multimedia  Product 
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During  the  initial  concept  development  phase  of  authoring,  each  multimedia  frame  was 
conceptualized  using  a  paper-based  drawing  that  identified  the  content  to  be  included.  Content 
could  include  background  images  or  colors,  text  fields,  user  interface  buttons  and  controls, 
timelines,  movies,  animations,  charts  and  figures,  sounds,  narration,  and  transitions  from  one 
frame  to  the  next.  Sequences  of  still  photographs  from  the  sled  tests  were  scanned  (digitized)  with 
a  flat  bed  scanner  and  saved  as  high-resolution  picture  files.  These  high-resolution  picture  files 
were  then  rescaled  to  provide  both  high  and  low-resolution  files  of  each  frame  for  multimedia 
authoring  and  publication.  A  detailed  inventory  of  multimedia  components  was  developed, 
including  file  names,  file  types,  file  sizes,  and  the  date  of  revision.  Opportunities  for  user 
interaction  that  support  communication  of  the  report  content  or  message  were  identified.  A 
prototype  of  the  multimedia  presentation  was  then  developed,  including  all  the  intended  frames, 
at  least  in  skeletal  form.  The  frames  were  then  completed  bit  by  bit.  Backgrounds  were  added, 
text  boxes  added,  and  other  elements  were  added.  Links  between  frames  were  established  and 
checked  out.  Multimedia  development  was  an  iterative  process,  consequently,  the  presentation 
was  developed,  tested,  refined,  and  retested  until  satisfactory. 

Movie  films  of  the  NVS  sled  runs  were  reviewed,  and  specific  sections  of  these  films 
were  selected  for  digitization.  The  AFFTC  photo  lab  was  investigated  as  a  source  to 
digitize  the  16-mm  motion  picture  films.  The  photo  lab  had  little  experience  digitizing 
motion  picture  film,  and  obtained  assistance  from  the  graphics  lab.  The  photo  lab  in 
conjunction  with  the  graphics  lab  used  an  Elmo  TRV-16G  to  obtain  digital  images  from  the 
film,  which  were  then  rendered  into  QuickTime®  movies  by  means  of  an  authoring 
program  (Macromedia  Director®).  However,  the  QuickTime®  movie  files  developed  by  the 
photo  lab  and  graphics  lab  were  too  low  in  resolution  (160  X  120  pixels)  to  be  satisfactory. 
The  Human  Factors  Branch  (TSFH)  used  videotape  copies  of  the  16  mm  film  and  a 
SNAZZI®  video  capture  card  to  produce  Motion  Picture  Experts  Group  (MPEG)  movie 
files  in  a  higher  resolution  (352  X  240  pixels)  that  provided  a  more  satisfactory  display. 
These  MPEG  movies,  however,  did  not  meet  the  project  objective  for  developing  shuttered 
video.  The  real  need  for  this  effort  was  to  make  high-resolution  (640  X  480  pixels)  movie 
files  on  a  frame  by  frame  basis.  Edwards  AFB  needs  to  have  the  capability  to  digitize 
16-mm  movie  film  on  a  frame  by  frame  basis  and  to  convert  the  resulting  images  into 
high-resolution  movie  files.  (LL14) 

In  general,  people  are  quite  intolerant  of  poor  sound  quality  in  multimedia  presentations 
(Reference  7).  This  finding  would  suggest  that  specialized  sound  rooms  were  desirable  for 
recording  narration.  However,  a  commercial  sound  room  was  not  available,  and  the 
recordings  were  done  during  periods  when  there  was  minimal  distraction.  Narration  files 
were  developed  using  a  Labtec  C-324  noise-canceling  headset  with  boom  microphone  in 
conjunction  with  the  sound  recording  capabilities  of  Microsoft  Windows  98.  Narration  was 
saved  as  .wav  (sound  file  format)  files,  and  was  edited  with  an  editing  program  packaged 
with  a  Diamond  Multimedia  Sonic  Impact  S90  sound  card.  Monophonic  sound  recordings 
were  found  to  be  satisfactory  for  narration.  (LL15) 

An  authoring  program  called  IMSI  Multimedia  Fusion  (Reference  11)  was  selected  as 
the  most  desirable  of  the  alternatives  tested  for  publishing  the  interactive  multimedia 
component.  Multimedia  Fusion  produced  a  product  with  rapid  screen  displays,  good  user 
interactivity,  and  reliable  functioning  on  the  computers  tested.  However,  a  file  called 
CNCS232.dll  had  to  be  added  in  the  multimedia  folder  on  the  CD-ROM.  The  multimedia 
product  then  ran  without  requiring  software  installation,  thus  meeting  that  specific  project 
objective.  All  software  needed  for  display  of  the  multimedia  presentation,  including 
DLL  files,  should  be  included  on  the  published  CD-ROM.  (LL16)  However,  authoring  of 
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IMSI  Multimedia  Fusion  was  time-consuming,  especially  for  a  novice  author,  because  of  the 
many  functions  and  options.  Although  many  Multimedia  Fusion  features  were  used,  there 
remained  many  potentially  useful  programming  features  that  were  not  used  during  this 
project.  For  example,  no  use  was  made  of  animations,  synchronized  narration  and  animation, 
synchronized  movies  and  narration,  or  hyperlinks  to  the  World  Wide  Web. 

Product  Verification: 

Verification  of  multimedia  products  for  electronic  publication  was  especially  important 
because  multimedia  products  contained  numerous  hyperlinks,  different  types  of  multimedia 
files,  and  because  the  products  would  be  displayed  on  a  wide  variety  of  computers  each 
having  different  hardware,  different  software,  different  capabilities,  and  different  settings. 
Verification  testing,  often  called  ‘beta  testing’,  should  be  performed  on  all  multimedia 
products  intended  for  publication.  (LL17)  Appendix  F  contains  a  survey  worksheet  that 
may  be  used  in  support  of  beta  testing.  An  HTML  validation  test  should  be  performed  on 
all  HTML  documents  that  are  intended  for  publication  (References  12,  13,  and  14). 
(LL18) 

The  use  of  PDF  files  with  the  Acrobat  reader  for  viewing  text  documents  was 
demonstrated  as  a  good  choice,  producing  an  excellent  and  user-friendly  publication  product 
acceptable  to  DTIC.  The  PDF  was  designed  for  electronic  document  publication  and 
computer  viewing  rather  than  for  word  processing.  An  Adobe  Acrobat  viewer  had  to  be 
installed  on  the  user  computer,  but  this  viewer  provided  fast  and  convenient  navigation 
within  documents,  navigation  across  files,  and  capability  to  zoom  the  display  to  different 
levels  of  magnification.  The  user  could  select  to  use  bookmarks  within  the  document  for 
navigation,  or  could  select  to  use  tiny  thumbnails  of  each  page  for  navigation.  The  user  could 
navigate  by  mouse  or  by  keyboard  commands.  Altogether,  the  Acrobat  reader  provided  rapid 
and  satisfying  methods  for  navigating  text  documents  and  accessing  the  multimedia 
presentation  when  published  on  computer  media  (CD-ROM).  A  negative  feature  of  Adobe 
PDF  files  was  that  formatting  was  lost  when  copying  from  PDF  files  and  pasting  into 
Microsoft  Word  documents. 

Static  and  dynamic  HTML  products  were  evaluated  with  a  variety  of  Web  browsers, 
including  Microsoft  Internet  Explorer®,  Netscape  Navigator®,  and  the  Opera®  web  browser 
fhttD://www.operasoftware.comA.  These  tests  revealed  that  each  browser  had  different 
versions,  display  and  operating  characteristics,  plug-ins,  and  settings  on  user’s  computers. 
The  Microsoft  Internet  Explorer  browser  proved  to  have  the  least  difficulties  displaying 
dynamic  HTML  documents.  Browser  difficulties  were  compounded  with  various  authoring 
programs  implementations  of  static  and  dynamic  HTML.  Furthermore,  the  Astound  authoring 
program  Version  5.0  produced  HTML  coding  that  did  not  pass  HTML  standards  validation 
(References  5,  12,  and  13),  indicating  that  Astound  HTML  products  might  not  perform 
correctly  on  some  Web  browsers.  The  Astound  Version  5  also  had  problems  producing  self- 
run  product  files,  which  may  be  corrected  in  future  revisions.  Astound  Versions  6.0  and  later 
deserve  a  second  look,  especially  for  the  executable  file  product.  The  latest  versions  of  all 
software  considered  for  use  should  be  evaluated  for  meaningful  performance 
improvements.  (LL19)  Multimedia  Fusion  was  tested  and  was  found  to  be  a  powerful, 
capable,  and  bug-free  program  that  produced  a  self-running  product  that  ran  on  all  computers 
tested  which  had  a  suitable  MPEG  movie  viewer. 
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Technical  Report  Verification: 

The  multimedia  CD-ROM  development  project  was  accomplished  in  a  number  of  steps: 

1.  The  results  of  the  NVS  test  were  prepared  as  a  paper-based  TERS  document 
(Reference  1), 

2.  The  TERS  in  Microsoft  Word  was  developed  as  a  CD-ROM  publication,  using  extensive 
hyperlinks  and  including  high-resolution  picture  files  for  detailed  viewing  or  subsequent  use, 

3.  The  TERS  was  developed  into  TR  format  as  a  paper-based  document, 

4.  The  TR  document  was  converted  from  Microsoft  Word  to  Adobe  PDF  files, 

5.  The  multimedia  was  developed  as  a  self-running  interactive  multimedia  file  using  IMSI 
Multimedia  Fusion  as  the  authoring  program, 

6.  The  text  documents  and  multimedia  presentations  were  supplemented  with  high- 
resolution  picture  files, 

7.  All  files  in  the  configuration  for  electronic  publication  were  put  on  the  computer  having 
the  CD  recorder, 

8.  Once  in  publication  configuration,  all  document  links  were  checked  (LL21  and  LL22), 
and 

9.  The  TR  was  published  on  CD-ROM  (Reference  1 5). 

The  CD-ROM  mastering  was  accomplished  using  Adaptec  Easy  CD  Creator  Standard 
Edition®  software  in  conjunction  with  a  Smart  and  Friendly®  CD-R4012.  When  the  Adaptec  CD 
mastering  program  was  started,  a  title  for  the  CD  was  added,  the  recording  option  of  ‘Data  CD’ 
was  selected,  the  files  selected  to  record,  and  the  CD  creation  process  started.  The  mastering 
program  performed  a  test  and  recorded  the  CD  without  further  intervention.  The  mastering 
process  took  about  one-half  hour  at  4X  recording  speed.  High-quality  blank  CD-ROM  media 
should  be  selected  to  ensure  media  longevity  as  well  as  playback  compatibility  with  different 
CD  drives  and  computer  systems.  (LL20)  Blank  CD  media  differed  in  the  types  of  dyes 
used  as  well  as  the  physical  structure  of  the  CD  including  scratch-resistant  coating.  The 
phthalocyanine  dye  used  in  Ricoh  Platinum®  CD  media  claimed  virtually  100%  hardware 
compatibility  and  an  expected  life  of  over  200  years. 

Viewing  the  PDF  text  documents  required  that  the  user’s  computers  have  Adobe  Acrobat 
Reader  installed.  Most  computer  users  had  this  program  installed,  but  for  those  who  did  not,  this 
program  was  provided  on  the  CD-ROM.  All  software  needed  for  display  of  the  multimedia 
presentation,  including  DLL  files,  should  be  included  on  the  published  CD-ROM.  Playback 
capability  for  MPEG  movies  was  required  of  the  user’s  computer.  For  computers  without  MPEG 
movie  playback  capability,  the  ‘ReadMe’  file  referenced  a  software  decoder  that  could  be 
downloaded  and  installed  by  the  user  ( www.visiblelight.com/mall/onstapft/inHpy  htpA 
A  ‘ReadMe.txt’  file  on  the  CD-ROM  should  specify  user  requirements  for  computer 
hardware  and  software.  (LL21) 
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The  CD-ROM  itself  should  be  labeled  with  the  title,  TR  number,  distribution  statement, 
security  classification,  publisher,  and  report  sign-off  date.  Archival-quality  CD-ROM 
labels  should  be  used  to  hold  up  under  years  of  expected  usage.  (LL22)  The  published 
CD  label  should  be  included  on  the  CD-ROM  in  printable  format,  such  as  Adobe  PDF, 
to  support  disc  copy  for  duplication  or  replacement.  (LL23)  The  electronic  publication 
included  front  and  back  paper  covers  with  a  few  pages  of  descriptive  material  along  with 
the  CD-ROM  in  a  plastic  sleeve.  The  electronic  publication  product  looked  like  a  very  thin 
conventional  paper-based  report.  When  desired,  destruction  of  CD-ROMs  was 
accomplished  by  cutting  them  into  several  pieces  before  trashing. 

Technical  Report  Application: 

Table  4  summarizes  meeting  of  project  objectives.  The  overall  objective  of 
demonstrating  the  application  and  worth  of  multimedia  methods  and  electronic  publication 
for  testing  and  reporting  was  met.  The  electronically  published  technical  report,  including 
multimedia,  was  well  received  by  the  customer.  Electronic  publishing  on  CD-ROM, 
including  interactive  multimedia,  should  be  considered  for  future  Air  Force  Flight 
Test  Center  technical  publications.  (R2)  The  project  objective  of  using  multimedia 
for  data  collection  and  analysis  was  not  met  because  of  inadequate  test  data.  The 
inadequate  data  collected  during  the  NVS  test  resulted  from  the  fact  that  proper  data 
collection  requirements  were  not  specified  during  the  test-planning  phase.  Data  analysis 
with  this  inadequate  data  was  attempted  ex  post  facto  without  success.  Many  lessons  were 
learned  to  guide  future  efforts,  and  lessons  learned  information  is  presented  in  Appendices 
A  and  B. 


Table  4 

PROJECT  OBJECTIVES  AND  ATTAINMENT 


Project  Objectives 

Objective  Met 

Overall  applicability  of  electronic  publishing 
for  technical  reporting. 

Yes,  electronic  product  was  well  received. 

1.  Multimedia  as  part  of  analytic  process. 

No,  necessary  test  planning  not  performed. 

2.  Multimedia  procedures  to  support: 
Frame-by-frame  (shuttered)  video, 
Time-space  positioning  information,  and 
Synchronized  plots  and  movies. 

No,  ESSAS  camera  system  not  suitable. 

No,  Test  data  not  planned  for  this  analysis. 
No,  TSPI  analysis  not  performed. 

3.  Zoom-in  visual  display  capability. 

Yes,  PDF  viewer  has  zoom  feature. 

4.  Work  with  wide  variety  of  user  computers. 

Yes,  PDF  and  MPEG  viewers  required. 

5.  Product  acceptable  to  Defense  Technical 

I  Information  Center. 

Yes,  for  both  PDF  and  multimedia  files.  I 

6.  Product  durable  and  sustainable  over  time. 

Yes,  CD  durable  and  PDF  viewer  was  j 

included.  f 

Notes:  1 .  ESSAS  -  Edwards  Stores  Separation  Analysis  System 

2.  TSPI  -  time-space  positioning  information 

3 .  PDF  -  portable  document  format 

4.  MPEG  -  Motion  Picture  Experts  Group 

5 .  CD  -  compact  disc 
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Many  advantages  were  found  for  the  Adobe  PDF  from  a  user  perspective.  The  text  and 
pictures  displayed  quickly  and  clearly.  The  Acrobat  Reader  was  developed  as  a  document  viewer, 
and  had  many  features  to  support  the  user.  The  Acrobat  Reader  had  separate  functions  for 
navigating  within  a  document  and  for  navigating  between  documents  via  hyperlinks  (an 
extremely  desirable  feature).  Document  navigation  could  be  achieved  through  mouse  clicks, 
the  Home,  End,  PageUp  and  PageDown  keys,  and  the  arrow  keys.  The  user  could  find 
his/her  way  through  the  document  by  means  of  bookmarks  (like  a  table  of  contents)  or  thumbnails 
(tiny  pictures  of  each  page).  Both  of  these  methods  had  their  own  special  advantages.  The 
PDF  viewer  permitted  pages  to  be  viewed  in  almost  any  size  from  thumbnails  up  to  800X 
magnifications.  The  mouse  could  be  used  to  slide  the  image  around  the  display  screen  to  locate 
and  display  areas  of  interest.  These  multiple  document  viewing  capabilities  satisfied  the  project 
objective  of  zoom-in  capability  to  areas  of  special  interest.  The  Acrobat  Reader  permitted  text 
to  be  searched  for  specific  words  or  topics.  Selected  pages  or  the  entire  PDF  document  could 
be  printed.  The  text  document  display  and  printed  format  did  not  vary  depending  upon  the  printer 
or  printer  settings  of  the  user’s  computer.  Adobe  PDF  should  be  considered  for  electronic 
publishing  of  text  files  because  the  Acrobat  viewer  facilitated  navigation  through  the 
document  and  retained  the  original  format  for  viewing  as  well  as  printing.  (R3)  Text  could 
be  copied  and  inserted  into  other  applications,  however  formatting  was  lost  when  copying  from 
PDF  files  and  pasting  into  Microsoft  Word  documents.  Portions  of  documents  expected  to  be 
copied  and  pasted  into  other  documents  should  be  included  in  the  CD-ROM  in  Microsoft 
Word  to  ensure  format  retention.  (LL24) 

Multimedia  Fusion  was  tested  and  was  found  to  be  a  powerful,  capable,  and  bug-free 
program  that  produced  a  self-running  product  that  ran  on  all  computers  tested  which  had 
a  suitable  MPEG  movie  viewer.  There  were  several  cases  where  the  user’s  computer  was 
not  configured  to  support  display  of  the  MPEG  movies,  but  MPEG  movie  support  was  a  stated 
user  system  requirement.  The  use  of  self-running  multimedia  files  should  be  used  because 
self-running  files  did  not  require  installation  on  the  user’s  computer,  and  multimedia 
display  was  not  dependent  upon  the  specific  browser  or  browser  settings  on  the  user’s 
computer.  (R4) 
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CONCLUSIONS  AND  RECOMMENDATIONS 

The  overall  objective  of  demonstrating  the  application  and  worth  of  multimedia  methods  and 
electronic  publication  for  testing  and  reporting  was  attained.  The  developed  technical  report  (TR) 
was  published  on  compact  disc-read-only  memory  (CD-ROM)  in  a  format  that  was  acceptable  to 
the  Defense  Technical  Information  Center  (DTIC)  for  electronic  publishing.  The  multimedia  TR 
was  well  received  by  the  user. 

2.  Electronic  publishing  on  CD-ROM,  including  interactive  multimedia,  should  be 
considered  for  future  Air  Force  Flight  Test  Center  technical  publications.  (Page  15) 

The  project  objective  of  using  multimedia  for  data  collection  and  analysis  was  not  achieved 
because  of  inadequate  test  data.  The  inadequate  data  collected  during  the  night  vision  system 
(NVS)  test  resulted  from  the  fact  that  data  collection  requirements  for  Edwards  Stores  Separation 
and  Analysis  System  (ESSAS)  application  were  not  specified  during  the  test-planning  phase. 

1.  Multimedia  should  be  considered  during  the  initial  test-planning  phase  to  ensure 
success  with  data  collection,  data  analysis,  and  results  publication.  (Page  5) 

The  use  of  Adobe  portable  document  format  (PDF)  files  with  the  Acrobat  reader  for  viewing 
text  documents  was  demonstrated  as  a  good  choice,  producing  an  excellent  and  user-friendly 
publication  product  acceptable  to  DTIC.  The  PDF  was  designed  for  electronic  document 
publication  and  computer  viewing  rather  than  for  word  processing.  Altogether,  the  Acrobat 
reader  provided  rapid  and  satisfying  methods  for  navigating  text  documents  and  accessing  the 
multimedia  presentation  when  published  on  computer  media  (CD-ROM). 

3.  Adobe  PDF  should  be  considered  for  electronic  publishing  of  text  files  because  the 
Acrobat  viewer  facilitated  navigation  through  the  document  and  retained  the  original 
format  for  viewing  as  well  as  printing.  (Page  16) 

Multimedia  Fusion  was  used  as  the  multimedia-authoring  program,  and  the  product  was 
developed  as  a  self-running  executable  file. 

4.  The  use  of  self-running  multimedia  files  should  be  used  because  self-running  files 
did  not  require  installation  on  the  user’s  computer,  and  multimedia  display  was  not 
dependent  upon  the  specific  browser  or  browser  settings  on  the  user’s  computer.  (Page  16) 
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REQUIREMENTS  FOR  EDWARDS  STORE  SEPARATION 
ANALYSIS  SYSTEM  DATA  ANALYSIS 

(Derived  from  Reference  8) 

Careful  planning  and  data  collection  system  calibration  must  be  performed  to  ensure  success 
of  subsequent  Edwards  Store  Separation  Analysis  System  (ESSAS)  analyses.  With  proper  input 
data,  ESSAS  results  may  achieve  an  accuracy  of  2  degrees  of  attitude  and  2  inches  of  movement 
throughout  the  entire  trajectory  of  the  object.  The  Range  may  be  tasked  to  perform  the  required 
calibrations.  The  following  planning  and  system  calibrations  are  required: 

1 .  The  exact  location  of  the  center  of  the  film  plane  for  each  camera  must  be  documented  in 
aircraft  coordinates.  Location  documentation  should  include  a  waterline,  buttline,  and  fuselage 
station  locations  in  conjunction  with  the  pointing  angle  of  the  camera.  Pointing  angles  are  given 
in  roll,  pitch,  and  yaw  as  measured  in  an  aircraft  type  coordinate  system. 

2.  The  camera/lens  system  must  be  calibrated.  The  camera/lens  calibration  consists  of 
mounting  the  camera/lens  onto  a  fixture  and  then  shooting  film  of  the  fixture.  The  film  of  the 
fixture  then  permits  calculation  of  the  depth-of-field  of  the  lens  as  well  as  the  determination  of  an 

inches  to  raster  counts’  conversion  factor.  The  inches  to  raster  counts  conversion  allow  the 
Range  to  match  an  image  size  on  film  accurately  to  its  physical  size.  The  Range  has  a  camera 
fixture  that  provides  all  needed  characteristics  for  this  calibration. 

3.  A  calibration  is  needed  of  the  location  of  each  object  to  be  analyzed  in  the  same  aircraft 
coordinate  system  as  measured  for  the  camera  locations.  It  is  necessary  to  know  where  the 
cameras  and  the  objects  are  in  the  same  reference  frame  before  any  movement  takes  place. 

4.  All  still  and  motion  picture  film  used  in  the  analyses  must  be  coded  with  Inter-Range 
Instrumentation  Group  (IRIG)  time. 
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APPENDIX  B 

LESSONS  LEARNED  SUMMARY 
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LESSONS  LEARNED  SUMMARY 


LLl.  If  ESS  AS  analysis  is  planned,  ESSAS  requirements  should  be  considered  during  the 
test-planning  phase.  (Page  5) 

LL2.  The  up-graded  ESSAS  capability  should  be  considered  as  a  resource  to  support 
multimedia  data  collection,  film  digitization,  and  data  analysis.  (Page  5) 

LL3.  No  single  approach  considered  for  text  and  multimedia  authoring  met  all  requirements, 
and  compromises  were  inherent  in  each  approach  selected.  (Page  8) 

LL4.  Low-resolution  picture  files  should  be  used  within  documents  to  provide  rapid  display, 
and  high-resolution  copies  of  these  pictures  provided  on  the  CD-ROM  for  printing,  high  quality 
display,  or  other  uses.  (Page  8) 

LL5.  Blank  pages  should  not  be  incorporated  into  electronic  publications  because  they  are 
annoying  for  the  reader.  (Page  8) 

LL6.  Single  column  format  should  be  used  for  electronic  publications  because  dual  column 
format  required  twice  as  much  mouse  or  keyboard  scrolling,  which  was  annoying  for  the  reader. 
(Page  8) 

LL7 .  Almost  all  text  editing  (except  for  inserting  or  revising  hyperlinks)  should  be  done  in 
the  Microsoft  Word  files  before  conversion  into  PDF  files.  (Page  8) 

LL8.  Adobe  Acrobat  Version  4.0  or  later  should  be  used  for  creating  PDF  files  from 
document  files.  (Page  9) 

LL9.  Hyperlinks  should  be  established  in  the  Adobe  PDF  files  as  they  are  configured  for 
publication  to  reduce  redundancy  of  verification  efforts.  (Page  9) 

LL10.  Hyperlinks  should  take  the  reader  to  the  exact  position  desired  within  a  document, 
rather  than  to  the  start  of  a  file.  (Page  9) 

LL11.  Trial  CDs  should  be  recorded  and  tested  on  different  computers  to  ensure  that 
hyperlinks  were  not  made  to  unintended  files  residing  in  other  locations  on  the  authoring 
computer.  (Page  9) 

LLl 2.  As  a  prerequisite  for  successful  electronic  publication,  all  user  computers  should  be 
configured  to  support  multimedia.  (Page  9) 

LLl 3.  Individual  computer  set-ups  were  found  to  vary  considerably  in  terms  of  browser 
selection,  browser  plug-ins,  video  cards,  and  display  settings,  all  of  which  affected  performance. 
(Page  9) 

LL14.  The  Base  needs  to  have  the  capability  to  digitize  16-mm  movie  film  on  a  frame  by 
frame  basis  and  to  convert  these  images  into  high-resolution  movie  files.  (Page  12) 

LLl  5.  Monophonic  sound  recordings  were  found  to  be  satisfactory  for  narration.  (Page  12) 
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LL1 6.  All  software  needed  for  display  of  the  multimedia  presentation,  including  DLL  files, 
should  be  included  on  the  published  CD-ROM.  (Page  12) 

LL17.  Verification  testing,  often  called  ‘beta  testing’,  should  be  performed  on  all  multimedia 
products  intended  for  publication.  (Page  13) 

LL18.  An  HTML  validation  test  should  be  performed  on  all  HTML  documents  that  are 
intended  for  publication.  (Page  13) 

LL19.  The  latest  versions  of  all  software  considered  for  use  should  be  evaluated  for 
meaningful  performance  improvements.  (Page  13) 

LL20.  High  quality  blank  CD-ROM  media  should  be  selected  to  ensure  media  longevity  as 
well  as  for  playback  compatibility  with  different  CD-drives  and  computer  systems.  (Page  14) 

LL21.  A  ‘ReadMe.txf  file  on  the  CD-ROM  should  specify  user  requirements  for  computer 
hardware  and  software.  (Page  14) 

LL22.  Archival  quality  CD-ROM  labels  should  be  used  to  hold  up  under  years  of  expected 
usage.  (Page  14) 

IT, 23.  The  published  CD  label  should  be  included  on  the  CD-ROM  in  printable  format,  such 
as  Adobe  PDF,  to  support  disc  copy  for  duplication  or  replacement.  (Page  14) 

LL24.  Portions  of  documents  expected  to  be  copied  and  pasted  into  other  documents  should 
be  included  in  the  CD-ROM  in  Microsoft  Word  to  ensure  format  retention.  (Page  16) 
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APPENDIX  C 


HARDWARE  AND  SOFTWARE  USED 


HARDWARE  AND  SOFTWARE  USED 


1.  Adaptec  Easy  CD  Creator  Standard  Edition  Version  3.5,  Adaptec,  Inc.,  691  S  Milpitas 
Blvd.,  Milpitas,  CA  95035.  http://www.adaptec.com/ 

2.  Adobe  Photoshop  4.0  LE,  Acrobat  PDF  Writer,  Acrobat  Distiller,  and  Acrobat  Exchange 
Version  3.01.  Adobe  Systems,  Inc.  345  Park  Ave.,  San  Jose,  CA,  95110-2704. 
http://www.adobe.com/ 

3.  Astound  Version  5.0,  Astound  Incorporated,  5155  Spectrum  Wy.,  Unit  5,  Mississauga, 
Ontario,  Canada  L4W5A1.  http://www.astound.com/frames/astound.html 

4.  Corel  PhotoPaint  7,  Corel  Corporation,  1600  Carling  Ave.,  Ottawa,  Ontario,  Canada 
K1Z8R7.  http://www.corel.com/ 

5.  CSE  3310  HTML  Validator,  http://htmvalidator.com/ 

6.  Dell  OptiPlex  GX1  computer  with  400  MHz  Pentium  II  processor  with  128  MB 
SDRAM,  32X  CD-ROM  reader,  ultra  wide  SCSI  controller,  8  MB  AGP  video  card,  and  equipped 
for  multimedia  presentation,  http://www.dell.com/ 

7.  Diamond  Multimedia  Sonic  Impact  S90.  Diamond  Multimedia  Systems,  2880  Junction 
Ave.,  San  Jose,  CA  95134-1992.  http://www.diamond.com/ 

8.  Labtec  C-324  noise-canceling  headset  with  boom  microphone,  http://www.labtec.com/ 

9.  IMSI  Multimedia  Fusion  Version  1.0,  IMSI  Corporation,  75  Rowland  Wy.,  Novato,  CA 
94945.  http://www.imisoft.com/mmfiision/index.html:  http://clickteam.com/ 

10.  Macromedia  Director  6.5,  Macromedia  Products,  http://www.macromedia.com/ 

11.  Microtek  ScanMaker  5  flatbed  scanner,  Microtek  Lab,  Inc.,  3715  Doolittle  Dr.,  Redondo 
Beach,  CA  90278-1226.  http://www.microtek.com/ 

12.  Opera  Web  Browser  Version  3.5.  http://www.opera.com/ 

13.  Serif  DrawPlus  4  software,  Serif  SPC,  P.O.  Box  803,  Nashua,  NH  03061. 
http://www.serif.com/ 

14.  Smart  and  Friendly  SCSI  CD-ROM  recorder,  Model  CD-R  4012,  Smart  and  Friendly, 
Inc.,  P.O.  Box  2312,  Chatsworth,  CA  91313-2312.  http://www.smartandfriendlv.com/ 

15.  SNAZZI  Video  Capture  Card  with  supporting  software  Version  2.12.  Dazzle  Multimedia 
Corporation,  45778  Northport  Loop  W.,  Fremont,  CA  94538. 
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APPENDIX  D 

ADOBE  ACROBAT  VERSION  3.01  SETTINGS  USED 
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ADOBE  ACROBAT  VERSION  3.01  SETTINGS  USED 


NOTE:  Adobe  Acrobat  Version  4.0  installed  correctly  without  additional  concern  for  these  settings. 
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APPENDIX  E 

GENERIC  MULTIMEDIA  PUBLICATION  GUIDANCE 
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GENERIC  MULTIMEDIA  PUBLICATION  GUIDANCE 

PRETEST  AND  TEST  ACTIVITIES 

Figure  El  provides  generic  guidance  for  future  multimedia  publishing  efforts.  Successful 
multimedia  efforts  should  start  during  the  initial  test-planning  phase.  Test  objectives  for 
multimedia  analysis  or  reporting  should  be  identified,  data  collection  requirements  should  be 
specified,  analytic  methods  using  multimedia  should  be  identified,  and  methods  for  displaying 
test  results  developed.  Coordination  with  the  Edwards  Stores  Separation  and  Analysis  System 
(ESSAS)  group  should  be  accomplished  if  appropriate.  Requirements  for  still  pictures,  movie 
pictures,  and  sound  recordings  should  be  identified,  using  Appendix  A  for  guidance.  During 
testing,  all  required  photographs,  movies,  and  audio  recording  should  be  obtained  and  these 
products  should  be  time  coded  with  Inter-Range  Instrumentation  Group  time.  The  ESSAS  group 
may  provide  support  for  16-mm  movie  film  digitization,  data  analysis,  and  plotting  of  test  results. 
Still  photographs  and  sound  recordings  may  be  digitized  in-house  or  through  contractor  support. 


PRETEST  ACTIVITIES 


TEST  ACTIVITIES 


Note:  1 .  ESSAS  -  Edwards  Stores  Separation  and  Analysis  System 
Figure  El  Generic  Pretest  and  Test  Activities 
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DEVELOPING  MULTIMEDIA  REPORTS 

Figure  E2  provides  generic  guidance  for  developing  multimedia  reports.  Normally,  the  text 
report  would  be  developed  in  Microsoft  Word  and  converted  to  Adobe  portable  document  format 
(PDF)  files.  Text  reports  could  then  be  supported  with  separate  multimedia  presentations  that 
would  be  associated  through  hyperlinks.  This  approach  would  continue  the  traditional  look  and 
structure  of  Air  Force  Flight  Test  Center  (AFFTC)  paper-based  reports  while  at  the  same  time 
introducing  the  newer  technologies  of  electronic  publication,  hyperlinks,  and  interactive 
multimedia.  The  combination  of  text  report  and  multimedia  would  then  be  recorded  on  compact 
disc-read-only  memory  (CD-ROM)  for  publication  and  distribution.  Because  of  the  newness  of 
electronic  publishing,  some  conventional  paper-based  reports  may  still  be  necessary. 


3 .  CD-ROM  -  compact  disc-read-only  memory 


Figure  E2  Generic  Plan  for  Developing  Multimedia  Reports 
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Figure  E2  also  illustrates  that  multimedia  products  produced  during  data  analysis  could  be 
included  in  the  final  report.  On  the  other  hand,  new  multimedia  topics  may  be  identified  and 
developed  for  reporting.  The  media  products  required  for  multimedia  presentation  should  be 
identified  and  catalogued.  Detailed  multimedia  authoring  procedures  are  discussed  below.  The 
multimedia  frames  should  be  verified  as  they  are  developed.  When  the  multimedia  is  completed, 
the  product  should  be  recorded  on  CD-ROM  and  checked  on  a  variety  of  computers  and  with  a 
variety  of  users. 

Technical  reports  are  typically  developed  in  Microsoft  Word.  During  the  example  publication 
effort,  the  PDF  was  found  to  be  superior  to  Word  for  electronic  publishing  as  well  as  being 
acceptable  to  DTIC.  All  document  editing  should  be  done  in  Word,  using  the  default  PostScript 
driver  used  for  the  PDF  driver,  as  only  minor  changes  may  be  performed  on  the  resultant  PDF 
files.  That  is,  hyperlinks  can  be  set  and  single  words  or  an  existing  line  could  be  deleted  or 
modified  in  minor  ways.  However,  major  editing  cannot  be  done.  Words  cannot  be  added  where 
none  existed  before,  line  height  or  spacing  cannot  be  changed,  and  word  wrap  or  paragraph 
formatting  cannot  be  accomplished.  A  parallel  copy  of  the  report  may  have  to  be  developed  for 
the  paper-based  publication  that  has  the  required  blank  pages  and  other  minor  differences.  All  the 
report  files  intended  for  electronic  publishing  should  be  assembled  on  the  computer  to  be  used  for 
recording  in  the  exact  folders  and  configuration  intended  for  publication.  All  document  links 
should  be  set  and  verified  within  this  publication-ready  configuration,  and  the  hard  drive  should 
be  defragmented  prior  to  recording.  A  point  to  consider  during  multimedia  verification  is  that 
hyperlinks  could  be  set  unintentionally  to  files  elsewhere  on  the  authoring  computer,  and  that 
verification  of  links  using  different  computers  is  essential  for  detecting  these  false  links. 
Recording  may  be  accomplished  using  Adaptec  EZ  CD  Creator  for  the  first  copy.  High  quality 
CD  media  should  be  considered  to  support  compatibility  and  longevity.  Blank  CD  media  vary 
considerably  in  terms  of  physical  construction,  chemistiy,  and  recording  and  playback 
compatibility.  The  ‘copy  CD’  function  may  then  be  used  to  make  additional  CDs.  The  completed 
discs  should  be  labeled  and  packaged  for  publication.  Archival  quality  CD  labels  and  label 
printing  should  be  used,  and  files  for  printing  duplicate  labels  should  be  included  on  the  CD, 
preferably  in  Adobe  PDF  format. 

STEPS  FOR  MULTIMEDIA  AUTHORING 

Figure  E3  presents  detailed  steps  for  multimedia  authoring,  which  have  been  modified  from 
Reference  7.  This  reference  book  provides  many  excellent  examples  of  multimedia  projects, 
including  conceptual  approaches,  frame  layouts,  presentation  styles,  GUIs,  the  hardware  and 
software  used  for  development,  the  multimedia  development  team,  and  the  amount  of  time 
required  for  development.  Since  every  project  is  different,  no  exact  procedure  can  be  developed. 
However,  the  steps  identified  provide  insight  into  the  overall  multimedia  authoring  process  as 
well  as  identifying  areas  for  concern. 
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NOTE:  Modified  from  Reference  7,  Web  Design  Wow!  Book. 

Figure  E3  Detailed  Steps  for  Multimedia  Authoring 

Identify  Users  and  Messages: 

The  first  step  for  multimedia  authoring  is  to  identify  the  users  and  the  messages  that  are  to  be 
communicated.  Technical  reports  have  many  potential  users,  such  as  high-level  decision-makers, 
program  managers,  contractors,  military  personnel  in  the  field,  and  AFFTC  test  engineers. 
General  types  of  information  to  be  communicated  with  technical  reports  include  test  objectives, 
test  methods,  test  results,  test  conclusions,  system  deficiencies,  and  recommendations  for  system 
improvement.  Specific  messages  or  topics  should  be  identified  where  multimedia  presentations 
may  be  used  to  maximum  advantage. 

Conceptualize  Multimedia  Solution: 

Identification  of  the  messages  to  be  communicated  establishes  goals  for  multimedia 
development.  Other  factors  to  consider  are  the  look  and  feel  of  the  presentation,  the  size  of  the 
presentation,  and  whether  the  presentation  should  provide  links  to  outside  sources  via  the  WWW. 
A  home  page  (or  leadoff  menu  page)  should  be  developed  to  tell  the  user  at  a  glance  what  types 
of  information  are  included  in  the  presentation  and  the  methods  (i.e.,  navigational  tools)  for 
accessing  this  information. 
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Determine  Multimedia  Content: 


Knowledge  of  the  messages  to  be  communicated  and  understanding  of  the  potential  users 
should  help  the  author  to  identify  specific  media  and  combinations  of  media  for  the  presentation. 
Data  analyses  performed  during  the  test  may  have  produced  movies,  sound  tracts,  and  still 
pictures  that  may  be  employed  in  the  multimedia  publication.  Some  multimedia  content  may 
have  to  be  developed  specifically  for  the  report.  The  type  and  extent  of  user  interaction  should  be 
considered. 

Choose  Display  Interaction  Tools: 

Interactivity  is  an  important  new  capability  for  electronic  publication.  Interactivity  should  be 
used  to  the  fullest  extent  to  support  active  user  involvement,  accessing  information, 
understanding  presentation  messages,  and  permitting  users  to  make  a  unique  path  through  the 
document.  Multimedia  presentations  may  include  sound,  motion,  graphics,  animation,  user 
interaction,  and  hyperlinks.  Other  multimedia  technologies  may  be  considered  for  technical 
publication,  and  these  features  include  text  that  moves  across  the  screen,  charts  that  grow  over 
time,  and  elegant  transitions  from  one  frame  to  the  next.  The  display  and  interaction  tools 
selected  for  technical  reporting  should  make  the  informational  content  more  accessible,  more 
understandable,  more  enjoyable,  and  better  remembered. 

Establish  Presentation  Style: 

Multimedia  presentations  should  be  an  appealing  and  rewarding  experience  for  the  user.  The 
style  and  ‘personality’  of  the  multimedia  should  reflect  the  character  of  the  sponsoring  agency, 
support  the  messages  to  be  communicated,  and  make  the  presentation  both  effective  and 
enjoyable.  The  presentation  should  have  a  degree  of  consistency  from  one  frame  to  the  next, 
reflecting  unity  of  design.  For  technical  reporting,  the  presentation  should  be  dignified  and 
professional,  yet  engage  the  user’s  attention,  interest,  and  desire  for  in-depth  study. 

Map-out  Organization  and  Navigation: 

Presentations  should  be  mapped-out  graphically,  so  that  the  entirety  of  the  presentation  can 
be  seen  at  a  glance.  Figure  3  is  the  flowchart  used  for  the  multimedia  technical  report  described 
within  this  document.  This  test  case  presentation  contained  81  frames,  although  size  should  not 
be  a  primary  consideration.  This  flowchart  describes  the  logical  structure  of  the  presentation  and 
the  logical  relationship  between  frames.  The  procedure  for  navigation  should  be  obvious  and 
straightforward.  In  general,  a  presentation  should  contain  a  ‘home  page’,  which  presents  the 
scope  of  the  presentation  and  navigational  tools  for  accessing  the  major  topic  areas.  In  general, 
the  number  of  navigation  options  for  each  page  should  be  limited  to  about  10,  and  all  information 
should  be  accessible  within  three  levels  of  depth.  The  multimedia  author’s  concept  of  the  path  to 
be  used  for  accessing  information  should  be  clear  to  the  user,  but  at  the  same  time  the  user  should 
be  able  to  determine  a  unique  path  to  be  taken.  That  is,  the  user  should  be  permitted  to  feel  in 
control  of  the  multimedia  experience. 

Storyboard  Presentation: 

Storyboarding  is  an  excellent  method  for  defining  the  specific  frames  (pages)  of  the 
presentation.  Objectives  of  stoiyboarding  are  to  ensure  that  the  content  is  complete  and  that  all 
the  planned  paths  through  the  frames  are  logical  and  complete.  The  presentation  should  start  with 
a  ‘home  page’  that  identifies  the  overall  topic  of  the  presentation,  provides  hyperlink  access  to  all 
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the  major  topics,  and  which  provides  a  point  to  which  the  user  can  always  return.  There  should 
not  be  any  dead-end  paths.  One  method  for  storyboarding  would  be  to  describe  each  frame 
(page)  on  a  sticky  note,  and  to  structure  these  sticky  notes  on  a  white  board  or  tabletop.  The  ease 
of  repositioning  and  changing  sticky  notes  is  a  major  advantage  of  this  approach.  The  sticky 
notes  could  be  repositioned,  modified,  added  to,  or  deleted.  The  multimedia  author  and  other 
reviewers  could  evaluate  different  possible  paths  through  the  material  and  determine  the  most 
desirable  structure  for  the  material. 

Conceptualize  Prototype  Including  User  Interface: 

Each  frame  identified  during  storyboarding  may  be  elaborated  in  full  size  in  the  aspect  ratio 
for  publication.  For  example,  the  display  format  for  the  test  case  multimedia  was  800  X  600 
pixels,  having  an  aspect  ratio  of  4:3.  One  approach  would  be  to  establish  a  generic  layout  grid  in 
template  format  that  could  accommodate  the  proposed  informational  content  as  well  as  the 
proposed  GUI.  Several  templates  could  be  developed  to  accommodate  different  types  of 
information,  such  as  motion  pictures,  text,  and  charts.  Templates  would  be  desirable  to  provide  a 
sense  of  design  unity  as  well  as  saving  authoring  work.  That  is,  templates  could  be  developed 
only  once,  and  these  templates  could  be  used  in  numerous  frames  to  provide  many  common 
features  such  as  background  images,  logos,  icons,  push  buttons,  and  mock  data  fields.  Unused 
navigational  controls  on  specific  pages  could  be  grayed  out  to  indicate  that  they  were  non- 
operational,  thus  keeping  the  positions  of  the  controls  the  same  on  each  multimedia  page. 
Templates  may  be  altered  as  desired  to  meet  special  needs.  Each  prototype  page  should  be 
refined  to  show  final  layouts,  including  text  boxes,  illustrations,  tables,  charts,  movies,  and 
animations.  The  layouts  should  identify  picture  and  movie  fields  in  terms  of  size  in  pixels. 
Timelines  and  transitions  from  one  frame  to  the  next  may  also  be  identified.  The  prototype 
frames  may  be  subjected  to  critical  review  and  revision  during  the  prototype  stage,  as  the 
prototyping  phase  is  the  time  to  make  changes  and  to  ensure  that  everything  is  in  order. 
Prototype  multimedia  presentations  are  a  major  creative  project  associated  with  multimedia 
development,  and  should  be  done  to  a  level  of  detail  that  multimedia  specialists  could  take  this 
information  in  conjunction  with  required  text,  visuals,  and  other  source  files  and  develop  the 
desired  multimedia  presentation. 

Assemble  Content: 

The  source  files  for  all  the  multimedia  content  to  be  employed  should  be  assembled  and 
cataloged.  Content  may  include  text,  visuals,  and  audio.  Additionally,  standards  may  be 
established  for  the  look  and  feel  of  the  presentation.  Colors,  typefaces,  icons,  figure  labeling,  and 
display  and  navigation  features  of  the  presentation  could  be  defined.  The  AFFTC  author’s 
instruction  (Reference  2)  identifies  standards  for  printed  reports,  but  this  document  is  not 
currently  applicable  to  multimedia.  In  the  future,  official  AFFTC  publication  standards  could  be 
developed  for  multimedia  publications  that  include  template  files. 

Digitize  and  Edit  Files: 

Multimedia  source  documents  must  be  digitized  for  computer  application.  Still  photographs 
could  be  scanned  with  a  scanner  and  turned  into  uncompressed  high-resolution  files,  such  as 
Windows  bitmap  (BMP)  or  tagged  image  file  format  (TIF)  files.  Initial  photo  editing  should  be 
performed  on  uncompressed,  high-resolution  picture  files.  Edited  high-resolution  picture  files 
then  may  be  rescaled,  compressed,  and  optimized  for  computer  display.  Picture  file  formats  often 
used  for  computer  display  include  Joint  Photographic  Experts  Group  (JPG)  files  and  Graphics 
Interchange  Format  (GIF)  files.  Motion  picture  film  may  be  digitized  by  the  ESSAS  group  and 
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data  analyses  and  data  plots  developed  as  well.  High  quality  computer  sound  cards  have  the 
hardware  and  software  needed  for  digitizing  analog  audio  sources  and  for  performing  editing 
operations.  The  sound  files  may  be  either  monophonic  or  stereophonic,  and  the  digital  files  may 
be  in  saved  as  .wav  (sound  file  format)  files.  All  files  for  multimedia  development  should  be 
archived  so  that  they  will  not  be  lost  or  over-written. 

Create  Multimedia  Product: 

The  actual  electronic  multimedia  product  is  developed  with  a  multimedia-authoring  program. 
The  authoring  program  used  for  the  test  case  technical  report  was  IMSI  Multimedia  Fusion.  A 
program  called  Astound1  was  also  tried,  but  at  that  time  there  were  unresolved  problems  with  this 
product.  The  Astound  program  deserves  a  second  look,  as  the  observed  problems  may  be 
corrected  in  subsequent  versions.  However,  there  are  many  alternative  authoring  programs  that 
would  be  capable  of  producing  the  required  multimedia  products.  Program  development  is  done 
frame  (or  page)  at  a  time.  Each  multimedia  frame  may  be  filled  with  background  colors  or 
images,  text  blocks,  movies,  still  pictures,  animations,  sounds,  interactivity,  navigational  tools, 
and  hyperlinks.  The  internal  functionality  of  each  frame  may  be  checked  as  it  is  developed,  and 
groups  of  frames  may  be  assessed  together  leading  up  to  assessment  of  the  overall  product.  The 
output  of  most  multimedia  authoring  programs  may  be  saved  in  a  variety  of  formats,  but  the  test 
case  project  found  that  self-running  files  (EXE)  were  the  most  desirable  choice. 

Beta  Test  and  Refine  Multimedia  Product; 


The  multimedia  product  should  be  recorded  on  CD-ROM  and  tested  with  different  users  and 
different  computers.  The  process  of  testing  software  systems  for  readiness  for  release  is  called 
‘beta  testing’.  Guidance  for  beta  testing  is  provided  in  Appendix  F.  Many  aspects  of  the 
presentation  should  be  assessed,  including  the  description  of  system  requirements,  operating 
instructions,  the  mechanics  of  operation  including  navigational  controls  and  links,  the  overall 
impression  of  the  multimedia,  and  the  ability  of  the  multimedia  to  meet  its  objective  for 
communicating  technical  information.  The  processes  of  testing  and  revising  should  continue 
until  the  product  is  ready  for  publication. 
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APPENDIX  F 
BETA  TESTING  SURVEY 
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BETA  TESTING  SURVEY 


The  purpose  of  beta  testing  a  product  is  to  determine  how  the  program  works  on  different 
computers  and  with  a  wide  variety  of  users.  You  are  encouraged  to  try  all  possible  modes  of 
operation  and  to  try  to  ‘break  the  program’  by  anticipating  how  it  may  be  misused.  Document  all 
identified  problems  and  concerns  on  this  worksheet. 


Title  of  CD-ROM 

Version 

Tester  Name: 

Phone 

Operating  System 


CD-ROM  Speed 


CPU  Speed 


Video  Card 


RAM 


Monitor  Size 


BUG  REPORT 


Broken  Link  Broken  Video  Broken  Broken 

Sound  Graphic 


Broken  Chart  Dead-end 
Path 
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Rate  test  product,  circling  desired  rating. 

Marginally  Unsatisfactory 

Unsatisfactory  _ 

Very  Unsatisfactory  — . 


1.  Computer  requirements  clearly  stated 

2.  Disc  title  descriptive  of  actual  contents 

3.  Running  of  program  on  my  computer 

4.  Intuitiveness  of  navigational  tools 

5.  Capability  of  navigational  tools 

6.  Simplicity  of  navigation  through  document 

7.  Freedom  from  culls  or  dead-end  paths 

8.  Accessibility  of  information 

9.  Appropriate  use  of  visual  displays 

10.  Visual  display  speed  on  computer 

1 1 .  Movie/animation  quality 

12.  Quality  of  audio  sounds  and  narration 

13.  Appropriate  use  of  sounds/audio 

14.  Learning  supported  through  audio 

15.  Appropriate  use  of  narration 

16.  Appropriate  amount  of  interactivity 

17.  Content  supported  by  interactivity 

18.  Learning  supported  through  interactivity 

19.  Interaction  used  to  increase  involvement 

20.  Ample  opportunity  to  set  own  path 

21.  Presentation  scope  defined  on  home  page 

22.  Appropriate  total  amount  of  information 

23.  Appropriate  presentation  sequences 

24.  Presentations  at  appropriate  level  of  detail 

25.  Appropriate  presentation  style 

26.  Content  supported  by  movies/animation 

27.  Interest  augmented  by  movies/animations 

28.  Interest  sustained  throughout  presentation 

29.  Quality  of  product  integration 

30.  OVERALL  PRODUCT  ASSESSMENT 


1  2  3 
1  2  3 
1  2  3 
1  2  3 
12  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 
1  2  3 


-  Marginally  Satisfactory 

j—  Satisfactory 

j-Very  Satisfactory 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5'  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 

4  5  6 


Dislikes 

Return  to:  _  E-Mail:  _  Phone: 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


Abbreviation 

Definition 

Units 

BMP 

bitmap 

— 

CD 

compact  disc 

— 

CD-RW 

compact  disc-rewritable 

— 

CD-ROM 

compact  disc-read-only  memory 

— 

DLL 

dynamic  link  library 

— 

DTIC 

Defense  Technical  Information  Center 

— 

DVD 

digital  versatile  disc 

— 

ESSAS 

Edwards  Stores  Separation  Analysis  System 

— 

EXE 

executable  program  file 

— 

GIF 

graphics  interchange  format 

— 

GUI 

graphic  user  interface 

— 

HTML 

hypertext  markup  language 

— 

IRIG 

Inter-Range  Instrumentation  Group 

— 

JPG 

Joint  Photographic  Experts  Group 

— 

JON 

job  order  number 

— 

LAN 

local  area  network 

— 

MB 

megabyte 

— 

MPEG 

Motion  Picture  Experts  Group 

— 

NVS 

night  vision  system 

— 

PDF 

portable  document  format 

— 

QDC 

quick  disconnect 

— 

RAM 

random  access  memoiy 

— 

RTF 

rich  text  format 

— 

SCSI 

small  computer  system  interface 

— 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS  (Concluded) 


Abbreviation 

Definition 

Units 

SGML 

standard  generalized  markup  language 

— 

SPO 

System  Program  Office 

— 

TERS 

test  and  evaluation  results  sheet 

Test  PAES 

test  planning,  analysis,  and  evaluation  system 

— 

TIF 

tagged  image  file  format 

— 

TR 

technical  report 

— 

TSPI 

time-space  positioning  information 

— 

VHS 

video  home  system 

— 

.wav 

wave  sound  file  format 

— 

WWW 

world  wide  web 

— 

X 

times  (i.e.,  multiplication) 

■ _ 
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